Skip to content

fix: replace_string_in_file instructions for claude #538

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

agreaves-ms
Copy link

Reminder Instructions

  • Provides more detailed instructions to help Claude use the replace_string_in_file tool correctly.
    • Claude seems to struggle with:
      • Un-escape characters in original files
      • Mis-matched indentation and whitespace in original files
      • Making non-changing updates to existing code
      • Making updates to code that it modified previously.

Notes:

  • I avoided merging in these instructions into the SweBenchAgent instructions to avoid breaking the logic that exists specifically for SweBenchAgent. Instead, I made targeted adjustments to the existing instructions and merged similar instructions with the new ReplaceStringInFileToolInstructions.

Details:

Claude attempting to make changes to a fairly large file, resulting in failed replace_string_in_file tool calls and healStringReplace calls:

image

Updated Instructions

After these sorts of instructions (I forced them in my copilot-instructions.md file), Claude is much more successful at making its changes:

image

@connor4312
Copy link
Member

Thanks for the PR, do you have any data to quantify these changes?

@agreaves-ms
Copy link
Author

Thanks for the PR, do you have any data to quantify these changes?

Hey @connor4312, I only have the ability to test from my machine. I can try doing several more tests from my machine and collect the logs and then compose some metrics (with the instructions and without for large files and many files). This would probably be better if it could be distributed out to multiple agents to test and validate...

Subjectively, I've been using these instructions all day for very large files (>3k lines long markdown files) and have been having more success with these replaceStringInFileInstructions, than without previously.

These added instructions for replace_string_in_file mostly help out in two ways:

  1. Large files as attachments are attached summarized with isSummarized=true. Claude seems to think it has the whole file when trying to edit it. This change doesn't fix this problem completely (claude and other models assuming it has the complete context), but I would often see Claude only update the top portion of the file or always append to the very end of a large file (and then do nothing in the middle).
  2. The content of the file being updated with multiple overlapping replace_string_in_file. The content for <attachment>'s are not updated as the file is modified by the edit tools. This change helps Claude understand this and attempt to do a read_file again instead of having it try to piece together the edits (and failures) itself.

@jojo0770
Copy link

提醒說明

  • 提供更詳細的說明,幫助 Claude 正確使用 replace_string_in_file 工具。

    • 克勞德似乎在掙扎:

      • 取消原始文件中的轉義字符
      • 原始文件中的縮排和空格不匹配
      • 對現有程式碼進行不改變的更新
      • 對之前修改過的程式碼進行更新。

筆記:

  • 我避免將這些指令合併到 SweBenchAgent 指令中,以免破壞 SweBenchAgent 固有的邏輯。相反,我對現有指令進行了有針對性的調整,並將類似的指令與新的指令合併ReplaceStringInFileToolInstructions

細節:

Claude 嘗試對相當大的檔案進行更改,導致 replace_string_in_file 工具呼叫和 healStringReplace 呼叫失敗:

image ### 更新說明 經過這些指令(我在 copilot-instructions.md 檔案中強制執行它們),Claude 在進行更改時更加成功: image

Uploading Wuta_20250811_175438_865_777.jpg…

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants